Method and system for predicting shape of human body after treatment

ABSTRACT

To have convenient and highly precise prediction of the shape of a human body after a treatment by calculation processing. 
     It includes a step of extracting a feature vector Fnew from face data of a patient as an evaluation subject, a step of selecting a plurality of case patients having feature vectors Fpre(i), extracted from the face data of a plurality of previous patients, that have a short distance from the feature vector Fnew, or a step of selecting a similar case class having a cluster center Gpre(l) that has a short distance from the feature vector Fnew, a step of obtaining pre-orthodontic facial shape models Hpre(i) and a post-orthodontic facial shape models Hpost(i) in which the faces of the selected previous case patients before and after a treatment have been normalized, a step of obtaining a facial shape model Hnew in which the face of the patient as an evaluation subject has been normalized, and a step of obtaining a three-dimensional predicted facial shape model Hprd as predicted after orthodontic treatment, by modifying the facial shape model Hnew of the patient as an evaluation subject, using a vector average difference AVEpost−AVEpre between the pre-treatment and post-treatment facial shape models of case patients having a facial shape similar to that of the patient as an evaluation subject.

TECHNICAL FIELD

The present invention relates to a method for predicting a shape of ahuman body after a treatment by arithmetic calculation processing, and asystem therefor. In particular, the present invention relates to amethod for predicting, with high precision, a shape of specific partlike a facial shape of a patient after an orthodontic treatment orbreast after operation, for example, and a system therefor.

BACKGROUND ART

Human face exhibits a strong influence on obtaining emotionalsatisfaction that he/she is socially accepted. Furthermore, the facialexpression plays an important role as a non-verbal communication meansfor performing delivery of emotion or thought in social life. Due tosuch reasons, in an orthodontic dental treatment of modern days,improvement of a shape of facial soft tissue is recognized as one of theimportant purposes of the treatment from the standpoint of socialpsychology.

For example, when a treatment plan is made by a dentist for a patienthaving malocclusion, in order to suitably determine the treatment planlike extraction or nonextraction, or need of a surgical operation or acamouflage treatment (treatment not associated with surgical operation),or the like, it is essential to evaluate objectively thethree-dimensional facial shape of a patient, and also predict the facialshape after the treatment.

Conventionally, prediction of a facial change after an orthodonticdental treatment is carried out based on profiles of a hard tissue(teeth skeleton) and a soft tissue (muscle and skin) of a patient beforeorthodontic treatment in which a head part X ray image (referred to as a“cephalogram” or simply as a “cephalo”) is shown. For example, nowwidely available is a software which carries out an image processingdisplay like move of a soft tissue following the move of a hard tissue,when the hard tissue is moved on a two-dimensional cephalo imagedisplayed on a monitor, and thus enables visualization and simulation ofa lateral view expected after the treatment.

However, the prediction algorithm using cephalo of a related art isconstituted on the premise that there is a simple correlation betweenthe movement amount of a hard tissue like tooth or jaw bone and themovement amount of a soft tissue like skin, and the correlation constantis also set based on subjective view or experiences of a professionalclinician or the like. Due to such reasons, a deviation in the result ofpredicted facial change exists among healthcare workers, and it is not atechnique of which prediction precision is guaranteed in quantitativeand objective sense.

For example, disclosed in Patent Document 1 is a method for predictingfacial appearance in terms of post-operative front view based on afrontal head part X ray image of a patient before operation in asurgical orthodontic treatment for a patient having jaw deformity, and ageneral photographic image of a face of the patient.

CITATION LIST Patent Document

Patent Document 1: JP 2014-171702 A

SUMMARY OF THE INVENTION Problem to be Solved by the Invention

An object of the present invention is to provide a technique allowingquantitative evaluation of a three-dimensional shape of a human bodybefore and after a treatment and contribution to the suitabledetermination of a treatment plan for the patient in view of previouscases.

Means for Solving Problem

To solve the problems described above, the present invention provides amethod for predicting a shape of a human body after a treatment byarithmetic calculation processing including a step of extractingmulti-dimensional case feature vectors from pre-treatment case shapedata obtained from a plurality of patients who have received thetreatment, a step of extracting a multi-dimensional patient featurevector from the patient shape data obtained from a patient, a step ofcalculating a patient shape model normalized based on the patient shapedata of the patient, a step of selecting a similar case feature vectorthat is similar to the patient feature vector from the case featurevectors, a step of calculating pre-treatment and post-treatment similarcase shape models that are normalized based on case shape datacorresponding to the selected similar case feature vectors; a step ofcalculating a difference between the pre-treatment similar case shapemodels and the post-treatment similar case shape models, and a step ofmodifying the patient shape model with the difference and calculating ashape model as predicted after the treatment of the patient.

The present invention further provides a method for predicting a facialshape after an orthodontic treatment including a step of extractingmulti-dimensional pre-orthodontic feature vectors Fpre(i) which have, aselements, a plurality of feature variables that are set in advance for ahuman facial shape based on pre-orthodontic three-dimensional face dataof a plurality of patients who have received an orthodontic treatment, astep of extracting a multi-dimensional subject patient feature vectorFnew having the feature variables as elements based on three-dimensionalface data of a new patient contemplating an orthodontic treatment(patient as an evaluation subject), a step of selecting, in apredetermined case number k, case patients having the pre-orthodonticfeature vectors Fpre(i) that are related to a plurality of the patientswho have received the orthodontic treatment in which the selection ismade in order of the shorter distance from the subject patient featurevector Fnew, a step of calculating pre-orthodontic facial shape modelsHpre (i=i1, i2, . . . , ik) as facial shape models of the each casepatient in which the arrangements of feature points obtained frompre-orthodontic three-dimensional face data of the each selected casepatient are normalized, a step of calculating post-orthodontic facialshape models Hpost (i=i1, i2, . . . , ik) of the each case patient inwhich the arrangements of feature points obtained from post-orthodonticthree-dimensional face data of the each selected case patient arenormalized, a step of calculating a vector average AVEpre ofpre-orthodontic facial shape models Hpre (i=i1, i2, . . . , ik) of theeach selected case patient, a step of calculating a vector averageAVEpost of post-orthodontic facial shape models Hpost (i=i1, i2, . . . ,ik) of the each selected case patient, a step of calculating a facialshape vector average difference AVEpost−AVEpre that is obtained bysubtracting the vector average AVEpre of pre-orthodontic facial shapemodels from the vector average AVEpost of post-orthodontic facial shapemodels, a step of calculating a subject patient facial shape model Hnewas a facial shape model of the patient in which the arrangement offeature points obtained from the three-dimensional face data of thepatient as an evaluation subject is normalized, and a step ofcalculating a three-dimensional predicted facial shape model Hprd aspredicted after an orthodontic treatment of the patient as an evaluationsubject by modifying the facial shape model Hnew of the patient as anevaluation subject with the facial shape vector average differenceAVEpost−AVEpre of the each selected case patient.

The present invention further relates to a method for predicting afacial shape after an orthodontic treatment including a step ofextracting multi-dimensional pre-orthodontic feature vectors Fpre(i)which have, as elements, a plurality of feature variables that areselected in advance for a human facial shape based on pre-orthodonticthree-dimensional face data of a plurality of patients who have receivedan orthodontic treatment, a step of calculating, for classification ofcase data of the plurality of patients, case classes classified into aplurality of classes by carrying out clustering processing of a group ofpre-orthodontic feature vectors Fpre(i) of the plurality of patients,and cluster centers Gpre(l) of the each case class, a step of extractinga multi-dimensional subject patient feature vector Fnew having thefeature variables as elements based on three-dimensional face data of anew patient contemplating an orthodontic treatment (patient as anevaluation subject), a step of selecting, from a plurality of thecluster centers Gpre(l) after the clustering, a similar case classhaving a cluster center that has the shortest distance from the subjectpatient feature vector Fnew, a step of calculating pre-orthodonticfacial shape models Hpre (i=i1, i2, . . . , ik) as facial shape modelsof the each case patient in which the arrangements of feature pointsobtained from pre-orthodontic three-dimensional face data of the eachselected case patient belonging to the similar case class arenormalized, a step of calculating post-orthodontic facial shape modelsHpost (i=i1, i2, . . . , ik) of the each case patient in which thearrangements of feature points obtained from post-orthodonticthree-dimensional face data of the each selected case patient belongingto the similar case class are normalized, a step of calculating a vectoraverage AVEpre of pre-orthodontic facial shape models Hpre (i=i1, i2, .. . , ik) of each selected case patient belonging to the similar caseclass, a step of calculating a vector average AVEpost ofpost-orthodontic facial shape models Hpost (i=i1, i2, . . . , ik) ofeach selected case patient belonging to the similar case class, a stepof calculating a facial shape vector average difference AVEpost-AVEprethat is obtained by subtracting the vector average AVEpre ofpre-orthodontic facial shape models from the vector average AVEpost ofpost-orthodontic facial shape models, a step of calculating a subjectpatient facial shape model Hnew as a facial shape model of the patientin which the arrangement of feature points obtained from thethree-dimensional face data of the patient as an evaluation subject isnormalized, and a step of calculating a three-dimensional predictedfacial shape model Hprd as predicted after an orthodontic treatment ofthe patient as an evaluation subject by modifying the facial shape modelHnew of the patient as an evaluation subject with the facial shapevector average difference AVEpost−AVEpre of the each selected casepatient belonging to the similar case class.

According to the method for predicting a facial shape described above,it is also possible that the method further includes a step ofextracting multi-dimensional pre-orthodontic cephalo feature vectorsCpre(i) which have, as elements, a plurality of feature variables thatare set in advance for a human bone shape based on pre-orthodontic headpart X ray images of the plurality of patients who have received anorthodontic treatment, wherein, in the step for classifying case data ofthe plurality of patients, the clustering processing is carried out fora group of extended feature vectors V(i) in which the pre-orthodonticfeature vectors Fpre(i) and the pre-orthodontic cephalo feature vectorsCpre(i) of the plurality of patients are composited.

Furthermore, according to the method for predicting a facial shapedescribed above, it is also possible that the method further includes astep of extracting multi-dimensional pre-orthodontic cephalo featurevectors Cpre(i) which have, as elements, a plurality of featurevariables that are set in advance for a human bone shape based onpre-orthodontic head part X ray images of the plurality of patients whohave received an orthodontic treatment, and a step of extractingmulti-dimensional post-orthodontic cephalo feature vectors Cpost(i)based on post-orthodontic head part X ray images of the plurality ofpatients, in which, in the step for classifying case data of theplurality of patients, the clustering processing is carried out for agroup of extended feature vectors V(i) in which the pre-orthodonticfeature vectors Fpre(i), the pre-orthodontic cephalo feature vectorsCpre(i), and the post-orthodontic cephalo feature vectors Cpost(i) ofthe plurality of patients are composited. Furthermore, it is preferablethat the extended feature vector V(i) is V(i)=[Fpre(i), Cpre(i),Cpre(i)−Cpost(i)] including, as an vector element, the cephalo featurevector difference Cpre(i)−Cpost(i) between the pre-orthodontic treatmentand post-orthodontic treatment.

Furthermore, it is preferable that the method for predicting a facialshape described above further includes a step of incorporating a featurevector and/or a facial shape model obtained from the three-dimensionalface data of the patient as an evaluation subject to a database.

The present invention further provides a system for predicting a facialshape after an orthodontic treatment provided with a database and acalculation processing device for carrying out calculation processingbased on data memorized in the database, in which the database memorizesin advance at least multi-dimensional pre-orthodontic feature vectorsFpre(i) which have been extracted having, as elements, a plurality offeature variables that are set in advance for a human facial shape basedon pre-orthodontic three-dimensional face data of a plurality ofpatients who have received an orthodontic treatment, pre-orthodonticfacial shape models Hpre(i) in which the arrangements of feature pointsobtained from pre-orthodontic three-dimensional face data of theplurality of patients are normalized, and post-orthodontic facial shapemodels Hpost(i) in which the arrangements of feature points obtainedfrom post-orthodontic three-dimensional face data of the plurality ofpatients are normalized, and the calculation processing device isprovided with a means for extracting feature vector to extract amulti-dimensional subject patient feature vector Fnew having the featurevariables as elements based on three-dimensional face data of a newpatient contemplating an orthodontic treatment (patient as an evaluationsubject), a means for selecting similar case patient to select, in apredetermined case number k, case patients having the pre-orthodonticfeature vectors Fpre(i) in order of the shorter distance from thesubject patient feature vector Fnew, a means for typing facial shape tocalculate a vector average AVEpre of pre-orthodontic facial shape modelsHpre (i=i1, i2, . . . , ik) of the each selected case patient andcalculating a vector average AVEpost of post-orthodontic facial shapemodels Hpost (i=i1, i2, . . . , ik) of the each selected case patient, ameans for modeling facial shape to calculate a subject patient facialshape model Hnew as a facial shape model of the patient in which thearrangement of feature points obtained from the three-dimensional facedata of the patient as an evaluation subject is normalized, and a meansfor calculating a predicted facial shape model to calculate athree-dimensional predicted facial shape model Hprd as predicted afteran orthodontic treatment of the patient as an evaluation subject bycalculating a facial shape vector average difference AVEpost−AVEpre thatis obtained by subtracting the vector average AVEpre of pre-orthodonticfacial shape models from the vector average AVEpost of post-orthodonticfacial shape models, and modifying the facial shape model Hnew of thepatient as an evaluation subject with the facial shape vector averagedifference AVEpost−AVEpre of the each selected case patient.

Furthermore, a system for predicting a facial shape after an orthodontictreatment according to another embodiment of the present invention isprovided with a database and a calculation processing device forcarrying out calculation processing based on data memorized in thedatabase, in which the database memorizes in advance at leastmulti-dimensional pre-orthodontic feature vectors Fpre(i) which havebeen extracted having, as elements, a plurality of feature variablesthat are set in advance for a human facial shape based onpre-orthodontic three-dimensional face data of a plurality of patientswho have received an orthodontic treatment pre-orthodontic facial shapemodels Hpre(i) in which the arrangements of feature points obtained frompre-orthodontic three-dimensional face data of the plurality of patientsare normalized; and post-orthodontic facial shape models Hpost(i) inwhich the arrangements of feature points obtained from post-orthodonticthree-dimensional face data of the plurality of patients are normalized,and the calculation processing device is provided with a means forclassifying case class to calculate a case class classified into aplurality of classes by carrying out clustering processing of a group ofpre-orthodontic feature vectors Fpre(i) of the plurality of patients,and cluster centers Gpre(l) of the each case class, a means forextracting feature vector to extract a multi-dimensional subject patientfeature vector Fnew having the feature variables as elements based onthree-dimensional face data of a new patient contemplating anorthodontic treatment (patient as an evaluation subject), a means forselecting similar case class to select, from a plurality of the clustercenters Gpre(l) after the clustering, a similar case class having acluster center that has the shortest distance from the subject patientfeature vector Fnew, a means for typing facial shape to calculate avector average AVEpre of pre-orthodontic facial shape models Hpre (i=i1,i2, . . . , ik) of the each selected case patient belonging to thesimilar case class and to calculate a vector average AVEpost ofpost-orthodontic facial shape models Hpost (i=i1, i2, . . . , ik) of theeach selected case patient belonging to the similar case class, a meansfor modeling facial shape to calculate a subject patient facial shapemodel Hnew as a facial shape model of the patient in which thearrangement of feature points obtained from the three-dimensional facedata of the patient as an evaluation subject is normalized, and a meansfor calculating a predicted facial shape model to calculate athree-dimensional predicted facial shape model Hprd as predicted afteran orthodontic treatment of the patient as an evaluation subject bycalculating a facial shape vector average difference AVEpost−AVEpre thatis obtained by subtracting the vector average AVEpre of pre-orthodonticfacial shape models from the vector average AVEpost of post-orthodonticfacial shape models, and modifying the facial shape model Hnew of thepatient as an evaluation subject with the facial shape vector averagedifference AVEpost−AVEpre of the each selected case patient belonging tothe similar case.

According to the system for predicting a facial shape with the aboveconstitution, it is also possible that the database further memorizes inadvance multi-dimensional pre-orthodontic cephalo feature vectorsCpre(i) which have been extracted having, as elements, a plurality offeature variables that are set in advance for a human bone shape basedon pre-orthodontic head part X ray images of the plurality of patientswho have received an orthodontic treatment, and the means forclassifying case class is to perform clustering processing for a groupof extended feature vectors V(i) in which the pre-orthodontic featurevectors Fpre(i) and the pre-orthodontic cephalo feature vectors Cpre(i)of the plurality of patients are composited.

Furthermore, according to the system for predicting a facial shape, itis also possible that the database further memorizes in advancemulti-dimensional post-orthodontic cephalo feature vectors Cpost(i)which have been extracted based on post-orthodontic head part X rayimages of the plurality of patients who have received an orthodontictreatment, and the means for classifying case class is to performclustering processing for a group of extended feature vectors V(i) inwhich the pre-orthodontic feature vectors Fpre(i), the pre-orthodonticcephalo feature vectors Cpre(i), and the post-orthodontic cephalofeature vectors Cpost(i) of the plurality of patients are composited.Furthermore, it is preferable that the extended feature vector V(i) isV(i)=[Fpre(i), Cpre(i), Cpre(i)−Cpost(i)] including, as an vectorelement, the cephalo feature vector difference Cpre(i)−Cpost(i) betweenthe pre-orthodontic treatment and post-orthodontic treatment.

Furthermore, it is preferable that the system for predicting a facialshape further includes a means for incorporating case data toincorporate a feature vector and/or a facial shape model obtained from athree-dimensional face data of the patient as an evaluation subject to adatabase.

The present invention still further provides a method for predicting abreast shape after a treatment including a step of extractingmulti-dimensional pre-operative feature vectors Fpre(i) which have, aselements, a plurality of feature variables that are selected in advancebased on pre-operative three-dimensional breast shape data of aplurality of patients who have received an operational treatment, a stepof calculating case classes classified into a plurality of classes bycarrying out clustering processing of a group of pre-operative featurevectors Fpre(i) of the plurality of patients, and cluster centersGpre(l) of the each case class, a step of extracting a multi-dimensionalsubject patient feature vector Fnew having the feature variables aselements based on three-dimensional breast shape data of a patientcontemplating the treatment, a step of selecting, from a plurality ofthe cluster centers Gpre(l) after the clustering, a similar case classhaving a cluster center that has the shortest distance from the subjectpatient feature vector Fnew, a step of calculating pre-operative breastshape models Hpre (i=i1, i2, . . . , ik) as a breast shape model of theeach case patient in which the arrangements of feature points obtainedfrom pre-operative three-dimensional breast shape data of the eachselected case patient belonging to the similar case class arenormalized, a step of calculating post-operative breast shape modelsHpost (i=i1, i2, . . . , ik) of the each case patient in which thearrangements of feature points obtained from post-operativethree-dimensional breast shape data of the each selected case patientbelonging to the similar case class are normalized, a step ofcalculating a vector average AVEpre of pre-operative breast shape modelsHpre (i=i1, i2, . . . , ik) of each selected case patient belonging tothe similar case class, a step of calculating a vector average AVEpostof post-operative breast shape models Hpost (i=i1, i2, . . . , ik) ofeach selected case patient belonging to the similar case class, a stepof calculating a breast shape vector average difference AVEpost−AVEprethat is obtained by subtracting the vector average AVEpre ofpre-operative breast shape models from the vector average AVEpost ofpost-operative breast shape models, a step of calculating a subjectpatient breast shape model Hnew as a breast shape model of the patientin which the arrangement of feature points obtained from thethree-dimensional breast shape data of the patient as an evaluationsubject is normalized, and a step of calculating a three-dimensionalpredicted breast shape model Hprd as predicted after operation bymodifying the breast shape model Hnew of the patient as an evaluationsubject with the breast shape vector average difference AVEpost−AVEpreof the each selected case patient belonging to the similar case class.

Effect of the Invention

According to the present invention, a three-dimensional shape of a face,breast, and other specific human body part of a patient before and aftera treatment can be conveniently predicted with high precision accordingto arithmetic calculation processing in view of previous cases. As such,it can contribute to suitable determination of a treatment plan for thatpatient.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating the brief constitution of asystem for predicting a facial shape;

FIG. 2 is a flowchart illustrating the method for predicting a facialshape according to a first embodiment;

FIG. 3 is a diagram illustrating an example of selecting featureparameters that are selected from contour lines of a human face;

FIG. 4 is a longitudinal cross-section view illustrating an additionalexample of selecting feature parameters;

FIG. 5 is a transverse cross-section view illustrating an additionalexample of selecting feature parameters;

FIG. 6 is a transverse cross-section view illustrating an additionalexample of selecting feature parameters;

FIG. 7 is a flowchart illustrating the method for predicting a facialshape according to a second embodiment;

FIG. 8 is a conceptual diagram illustrating a state in whichmulti-dimensional feature vector spaces are clustered;

FIG. 9 is a diagram for explaining the clustering processing of a facialshape feature vector and selection of a similar case class;

FIG. 10 is a block diagram illustrating the brief constitution of asystem for predicting a breast shape as a third embodiment; and

FIG. 11 is a flowchart illustrating the method for predicting a breastshape according to the third embodiment.

MODE(S) FOR CARRYING OUT THE INVENTION First Embodiment

The first embodiment of the present invention is explained. In FIG. 1,the brief constitution of a system for predicting a facial shape afteran orthodontic dental treatment is exemplified. The method forpredicting a facial shape according to the present invention is mainlycarried out by arithmetic calculation processing by a calculationprocessing device 10 in the system for predicting a facial shape shownin FIG. 1, for example. However, all processings that are explainedhereinbelow are not necessarily carried out by a single calculationprocessing device 10. Namely, as long as the problem of the presentinvention is solved, it is possible that case data obtained from anothersystem or intermediate data prepared as database according to previousperforming of calculation processing (specifically, feature vector,facial shape model, or the like) is utilized by the calculationprocessing device 10 of the system, for example. Furthermore, it is alsopossible that part of the processing for handling a large data likehigh-dimensional vector calculation processing or three-dimensionalmodeling processing is carried out by using a high speed host computeron network.

The calculation processing device 10 is connected with a data storage 20with large capacity, an input device 30, and an output device 40. Thedata storage 20 can be, other than a hard disk or an optical diskdirectly connected to the calculation processing device 10, a dataserver in hospital which can be accessed by the calculation processingdevice 10 via a network. Furthermore, the data storage 20 can beinstalled on a wide area network, for example, a cloud data center. Inthe data storage 20, case data prepared as a database 21 including a Xray photographic image data or a three-dimensional face data of apatient, a facial shape feature vector or a facial shape models asintermediate data, and a predicted facial shape models as evaluationdata, or the like are stored. Furthermore, it is preferable that theaccess to the data storage 20 or the database 21 is limited only to aspecial person who is allowed to share the data (for example, primarycare physician).

The calculation processing device 10 is a computer device which performscalculation processing based on the data that are memorized in the datastorage 20 or the database 21. The input device 30 includes amanipulation input device like keyboard, mouse, touch panel, or thelike, for example. Furthermore, the input device 30 can be a devicewhich has a function of inputting data that have been acquired orprocessed by another system to the calculation processing device 10 viaan information memory medium or a network. The output device 40 includesan image display for three-dimensional visual display of predictedfacial shape data, for example. Furthermore, for providing intermediatedata like feature vector processed by the calculation processing device10 or evaluation data like predicted facial shape model to anothersystem, the output device 40 can be a memory drive for memorizing thosedata in an information memory medium or a communication output devicefor outputting those data to an outside via a network.

Furthermore, the system for predicting a facial shape is constitutedsuch that face photography data of a patient or three-dimensional facedata that are taken in a hospital examination room or the like aresupplied to the calculation processing device 10, either directly or viathe data storage 20 or the database 21. It is acceptable that those facedata of a patient are inputted to the input device 30 via an informationmemory medium, or inputted to the system via a hospital network, forexample. The system for predicting a facial shape may also include, as aconstitutional element, a digital camera 61 for taking a picture of apatient, a three-dimensional camera 62 for acquiring three-dimensionalface data of a patient, or a general three-dimensional measurementdevice like three-dimensional scanner or laser profiler.

Memorized in the database 21 of the data storage 20 is at least thethree-dimensional face data which have been obtained from image data inwhich pre-orthodontic treatment and post-orthodontic treatment faces ofplural patients previously received an orthodontic dental treatment arephotographed by the three-dimensional camera 62 or anotherthree-dimensional measurement device. Furthermore, the case dataincluding three-dimensional face data of each patient are classified foreach item like sex, age, treatment area, treatment method, or the likeof each patient, and prepared as a database. Due to such reasons, withthe calculation processing device 10, case data can be searched from thedatabase 21 by focusing on a patient as an evaluation subject for whichthe treatment plan is just set to be determined, and a specific patientgroup with similar case like same sex, same generation, same treatmentarea or the like.

Furthermore, according to the system for predicting a facial shape ofthis embodiment, it is also possible that a head part X ray image data(hereinbelow, referred to as a “cephalo”) of a patient are inputted inthe calculation processing device 10 and/or the data storage 20(database 21). When the system of the present invention uses a cephaloof a patient, it is possible that the calculation processing device 10is connected in an accessible manner to an X ray examination device(cephalo system) 63 via a hospital network, for example. It is alsopossible that, as case data, pre-orthodontic and post-orthodonticcephalo data of a patient who has already received an orthodontic dentaltreatment are kept in advance as database in the data storage 20.

The calculation processing device 10 is provided with, as a means forcalculation processing to be achieved by the calculation processing, ameans for extracting feature vector 110, a means for selecting similarcase patient 120, a means for modeling facial shape 130, a means fortyping facial shape 140, a means for calculating a predicted facialshape model 150, a means for reconstructing facial shape 160, and ameans for incorporating case data 170. A means for extracting featurevector 110 includes a means for extracting pre-orthodontic featurevector 111, a means for extracting post-orthodontic feature vector 112,and a means for extracting subject patient feature vector 113. A meansfor modeling facial shape 130 includes a means for modeling case patientfacial shape 131 and a means for modeling subject patient facial shape132.

The first embodiment of the method consisting of the above calculatingmeans for predicting a facial shape after an orthodontic dentaltreatment is explained in detail hereinbelow in view of the flowchart ofFIG. 2.

First, a means for extracting pre-orthodontic feature vector 111 is toextract multi-dimensional pre-orthodontic feature vectors (pre-treatmentcase feature vectors) Fpre (i=1. 2, . . . , p) which have, as elements,plural feature variables that are set in advance for a human facialshape based on pre-orthodontic three-dimensional face data (case shapedata) of a plurality (p people) of patients who have already received anorthodontic dental treatment (step S11).

The “three-dimensional face data” means three-dimensional coordinatedata of the entire face or a pre-determined area of a patient likedental occlusion part which is acquired by the three-dimensional camera62 or a general three-dimensional measurement device likethree-dimensional scanner. The second-dimensional and three-dimensionalface data may be memorized in the database 21, either as one case dataof a patient or after being associated with case data of each patient.Furthermore, the face data may also include data of exhibiting at leasttwo facial expressions of the same patient (exhibition of resting face,smiling face, or the like).

Herein, a means for extracting feature vector 110 (a means forextracting pre-orthodontic feature vector 111, a means for extractingpost-orthodontic feature vector 112, a means for extracting subjectpatient feature vector 113) is a calculating means for finallyextracting a multi-dimensional feature vector by using a featureparameters, which are geometric parameters showing the feature of ashape of a human face. The “feature parameter” indicates a geometricparameter which characteristically shows a shape of a human face, and itis selected in advance based on, for example, experiences or knowledgeof professional clinicians. Herein, the feature parameters or featurevariables are explained a little bit further.

In FIG. 3, examples of the feature parameters that are selected from thecontour lines of a human face are shown. As shown in FIG. 3, severalinflection points can be recognized from a shape of a human face. Thoseinflection points can be selected from a corner of a boundary of an eyeor a nose, the three-dimensionally most protruded area, or the mostconcave area. In the present specification, those inflection points arereferred to as a landmark and used for defining feature parameters.Furthermore, the landmark is not particularly limited to be aninflection point, as long as it can be geometrically defined like acenter point of a line which connects two inflection points, or thelike.

Furthermore, the contour line of a face can be extracted as follows.First, by using a calculation program customized for measurement of afacial shape from a frontal image of a face, face normals are calculatedfor each pixel of three-dimensional surface data. Furthermore, the angleformed between the z coordinate and the face normals of a human face iscalculated for each coordinate on face surface. Each coordinate pointhaving 60 degrees for the angle formed between the z coordinate and theface normals of a human face is extracted, for example, and the lineconnecting those points is used as a facial contour line. The angledefining the facial contour line is preferably an angle between 45degrees and 90 degrees.

One example of the feature parameter is a distance between landmarks.For example, feature parameter v1 shown in FIG. 3 is defined as adistance between eye rim Exs (|Ex−Ex|). Furthermore, like v3, it can bea distance between a line connecting landmarks (for example, lineconnecting the sidemost end Zy′ of a face to jaw protrusion point Gn)and a landmark (for example, cheek protrusion point Go′). Furthermore,another example of the feature parameter is an angle of the lineconnecting landmarks. For example, the angle of feature parameter v4 isdetermined based on the positional relationship among the sidemost endZy′ of a face, cheek protrusion point Go′, and the cheek.

Furthermore, the distance feature parameter can be a dimensionlessvalue. For example, the width (|Ch−Ch|/|Ex−Ex|) obtained bynormalization of the mouth angle width (|Ch−Ch|) with the distancebetween eye rims (|Ex−Ex|) can be employed as a feature parameter.Furthermore, a deviation in an average of plural values or a ratiocompared to the average can be also considered as a feature parameter.

Furthermore, as shown in FIGS. 4 to 6, plural feature parameters areselected from a cross-section which is based on three-dimensional dataobtained by photographing a specific area of a human face. Thecross-section is obtained by data processing based on anatomical systempoint measurement, after determining a three-dimensional coordinatesystem. FIG. 4 shows, as an example, yz cross-section that is obtainedby cutting a face of a testee along the line connecting eye rim Ex andmouth angle point Ch. For example, the angle (v7) in z axis direction ofmouth angle Ch having eye rim Ex as a start point, the angle (v8) ofcheek protrusion point P(Ex-Ch) on the cross-section having eye rim Exas a start point, the distance (v12) of a contour line between eye rimEx and mouth angle Ch, and the area (v13) closed by that contour line,and the like can be selected as a feature parameter.

As an additional example, xz cross-section that is obtained by cutting aface of a testee along the horizontal surface which passes through undernose point Sn is shown in FIG. 5. Similarly, in FIG. 6, xz cross-sectionthat is obtained by cutting a face of a testee along the horizontalsurface which passes through the most protruded nose point Pm isexemplified. As shown in those figures, the protrusion amount (v14, v18)in z direction of a face area, the angle of protrusion point (v16, v20),the protrusion amount (v17, v22, v23), the angle (v21) of a concavepart, or the like at various cross-section positions can be selected asa feature parameter. The cross-section characterizing a facial shape canbe, although not illustrated, also a cross-section which passes througha glabella point Gla, nose root point N, upper lip point Ls, lower lippoint Li, or jaw end point Sm, other than those described above.Furthermore, a difference or a ratio compared to the average z value ofa specific area can be added to a feature parameter.

A means for extracting feature vector 110 is to carry out processing formeasuring a feature variable which corresponds to each of plural featureparameters that have been selected and set. By a means for extractingfeature vector 110, thus measured a n-dimensional feature vector F=[v1,v2, v3, . . . , vn] having n feature variables as vector elements isextracted, for example. It is also possible that, from data obtained byphotographing different facial expressions like resting face and smilingface, two or more feature vectors can be extracted for a single patient.Furthermore, it is also possible to extract an exhibition amount vectorwhich has, as vector elements, a feature variables measured from facialexpression exhibition data as a difference between them.

In step S11, according to the aforementioned processing, a means forextracting pre-orthodontic feature vector 111 extracts, for example,pre-orthodontic feature vectors Fpre(1) [v1, v2, v3, . . . , vn],Fpre(2) [v1, v2, v3, . . . , vn], . . . , Fpre(p) [v1, v2, v3, . . . ,vn] which have n feature variables as elements, for example, frompre-orthodontic three-dimensional face data of p patients who havealready received an orthodontic dental treatment.

It is also possible that, according to the same processing as describedabove, a means for extracting post-orthodontic feature vector 112extracts n-dimensional post-orthodontic feature vectors (post-treatmentcase feature vectors Fpost(1) [v1, v2, v3, . . . , vn], Fpost(2) [v1,v2, v3, . . . , vn], . . . , Fpost(p) [v1, v2, v3, . . . , vn] frompost-orthodontic three-dimensional face data (case shape data) ofpatients who have already received an orthodontic dental treatment.

It is also possible that feature vectors Fpre(i) and Fpost(i) areextracted by using a deep learning method. The extracted pre-orthodonticfeature vectors Fpre (i=1, 2, . . . , p) and post-orthodontic featurevectors Fpost (i=1, 2, . . . , p) are incorporated, as case data foreach patient, to the database 21. Furthermore, when the pre-orthodonticfeature vectors Fpre(i) and post-orthodontic feature vectors Fpost(i)are already provided to the database 21, the calculation processingdevice 10 may carry out the subsequent processing in view of the data ofthe database 21 without performing processing for extracting thosefeature vectors.

Based on the same processing as described above, a means for extractingsubject patient feature vector 113 is to extract, from three-dimensionalface data (patient shape data) of a new patient contemplating anorthodontic treatment (referred to as a “patient as an evaluationsubject”), an n-dimensional patient feature vector Fnew [v1, v2, v3, . .. , vn] which has the same feature variables as elements (step S12).

A means for selecting similar case patient 120 is to select, in a casenumber k, similar case patients having the pre-orthodontic featurevectors Fpre(i) that are related to p patients who have received theorthodontic treatment in which the selection is made in order of theshorter distance (|Fpre(i)−Fnew|) from the subject patient featurevector Fnew extracted in step S12 (step S13). Herein, the “distance”between vectors can be any of Euclid distance or Manhattan distance.

The case number k selected in step S13 is a number which is determinedin advance by a professional clinician or the like. Furthermore,regarding step S13, it is preferable to select a case patient who has ashorter distance with a subject patient contemplating an orthodontictreatment in terms of pre-orthodontic feature vectors Fpre(i) that havebeen narrowed down to a patient having common sex, age, treatment area,or the like.

Meanwhile, the three-dimensional face data obtained by using thethree-dimensional camera 62 or the like include a different number ofdata obtained based on a face size or the like of each patient, and havedifferent position of an original point depending on standing positionor the like of a photographed patient. In this regard, to havequantitative comparison or statistical processing of a facial shape ofeach patient, the system for predicting a facial shape of thisembodiment is provided with a means for modeling facial shape 130 formodeling three-dimensional face data with normalized facial shape model(a means for modeling case patient facial shape 131, a means formodeling subject patient facial shape 132). A means for modeling facialshape 130 is to perform calculation processing for constructing athree-dimensional facial shape model which has been normalized byextracting pre-set anatomical feature points from three-dimensional facedata of a patient and arranging the feature points on a polygon withidentical number of points and identical phase geometric structure. Theshape model constructed by such method is generally referred to as a“homologous model”, and Homologous Body Modeling (HBM) program providedby National Institute of Advanced Industrial Science and Technology canbe used, for example.

A means for modeling subject patient facial shape 132 is to calculatesubject patient facial shape model (patient shape model) Hnew which isobtained by extracting anatomical feature points from three-dimensionalface data of a patient as an evaluation subject (patient shape data)contemplating an orthodontic treatment and normalizing the arrangementof the feature points with a homologous model, for example (step S14).

Furthermore, a means for modeling case patient facial shape 131 is tocalculate pre-orthodontic facial shape models (pre-treatment similarcase shape models) Hpre (i=i1, i2, . . . , ik) which are obtained bynormalizing the arrangements of the anatomical feature points obtainedfrom pre-orthodontic three-dimensional face data of each of k similarcase patients as selected in step S13 by the aforementioned modelingprocessing (step S15). Similarly, a means for modeling case patientfacial shape 131 is to calculate post-orthodontic facial shape models(post-treatment similar case shape models) Hpost (i=i1, i2, . . . , ik)which are obtained by normalizing the arrangements of the anatomicalfeature points obtained from post-orthodontic three-dimensional facedata of each of k similar case patients as selected in step S13 by theaforementioned modeling processing (step S16).

Herein, as case data, all of the pre-orthodontic facial shape modelsHpre(i) and post-orthodontic facial shape models Hpost(i) calculated forpatients may be incorporated to the database 21. Furthermore, when thepre-orthodontic facial shape models Hpre(i) and post-orthodontic facialshape models Hpost(i) of previous patients are already provided to thedatabase 21, the calculation processing device 10 may carry out thesubsequent processing in view of the model data of the database 21without performing the modeling processing.

Subsequently, a means for typing facial shape 140 is to calculate vectoraverage AVEpre of pre-orthodontic facial shape models Hpre (i=i1, i2, .. . , ik) of each of selected k similar case patients (step S17).Similarly, a means for typing facial shape 140 is to calculate vectoraverage AVEpost of post-orthodontic facial shape models Hpost (i=i1, i2,. . . , ik) of each of selected k similar case patients (step S18).

Subsequently, a means for calculating a predicted facial shape model 150is to calculate facial shape vector average difference AVEpost−AVEpre bysubtracting the vector average AVEpre of pre-orthodontic facial shapemodel from the vector average AVEpost of post-orthodontic facial shapemodel (step S19). In addition, a means for calculating a predictedfacial shape model 150 is to perform calculation for adding facial shapevector average difference AVEpost−AVEpre of each of selected k similarcase patients to subject patient facial shape model Hnew of a patient asan evaluation subject, and thus obtaining predicted three-dimensionalpredicted facial shape model Hprd (=Hnew+AVEpost−AVEpre) of a patient asan evaluation subject after an orthodontic treatment (step S20).

The predicted facial shape model Hprd obtained by step S20 is just roughexpression of anatomical feature points only that are predicted after anorthodontic dental treatment. As such, to have more faithfulreproduction of a predicted actual facial shape, a means forreconstructing facial shape 160 is preferably to reconstruct a predictedthree-dimensional face data by rearranging the anatomical feature pointsof predicted facial shape model Hprd in a three-dimensional face datacoordinate system of a patient as an evaluation subject (step S21).Accordingly, it becomes possible to display three-dimensionally alateral view of a face of a patient predicted after a treatment or thelike on the output device 40 like image display (step S22), and thus apatient is allowed to fully understand the effect of an orthodonticdental treatment while watching the image.

With a means for incorporating case data 170, three-dimensional facedata of a subject patient after the evaluation may be incorporated tothe database 21. Furthermore, with a means for incorporating case data170, pre-orthodontic subject patient feature vector Fnew extracted fromthe three-dimensional face data of a patient as an evaluation subject orsubject patient facial shape model Hnew obtained by modeling theanatomical feature points of three-dimensional face data may beincorporated to the database 21. It is also possible that, with a meansfor incorporating case data 170, the post-orthodontic treatmentthree-dimensional face data of the patient, the post-orthodontic featurevector and/or post-orthodontic facial shape model which is extractedfrom the post-treatment three-dimensional face data are incorporated tothe database 21. By accumulating those data in the database 21,precision of the facial shape prediction after a treatment of a futurepatient can be further enhanced.

Second Embodiment

Next, the second embodiment of the present invention is explained inview of the flowchart of FIG. 7. Furthermore, the method for predictinga facial shape according to the second embodiment can be carried out byusing the system shown in FIG. 1, similar to the first embodiment.

The calculation processing device 10 is provided with, as a means forcalculation processing to be achieved by the calculation processing, ameans for extracting feature vector 110, a means for classifying caseclass 180, a means for selecting similar case class 190, a means formodeling facial shape 130, a means for typing facial shape 140, a meansfor calculating a predicted facial shape model 150, a means forreconstructing facial shape 160, and a means for incorporating case data170.

A means for extracting feature vector 110 includes a means forextracting pre-orthodontic feature vector 111, a means for extractingpost-orthodontic feature vector 112, and a means for extracting subjectpatient feature vector 113.

A means for modeling facial shape 130 includes a means for modeling casepatient facial shape 131 and a means for modeling subject patient facialshape 132.

First, a means for extracting pre-orthodontic feature vector 111 is toextract n-dimensional pre-orthodontic feature vectors (pre-treatmentcase feature vectors) Fpre(1) [v1, v2, v3, . . . , vn], Fpre(2) [v1, v2,v3, . . . , vn], . . . , Fpre(p) [v1, v2, v3, . . . , vn] which have, aselements, plural feature variables that are set in advance for a humanfacial shape, from pre-orthodontic three-dimensional face data (caseshape data) of p patients, for example, who have already received anorthodontic dental treatment (step S31).

It is also possible that, according to the same processing as describedabove, a means for extracting post-orthodontic feature vector 112extracts n-dimensional post-orthodontic feature vectors (post-treatmentcase feature vectors) Fpost(l) [v1, v2, v3, . . . , vn], Fpost(2) [v1,v2, v3, . . . , vn], . . . , Fpost(p) [v1, v2, v3, . . . , vn] frompost-orthodontic three-dimensional face data (case shape data) ofpatients who have already received an orthodontic dental treatment,similar to above.

It is also possible that feature vectors Fpre(i) and Fpost(i) areextracted by using a deep learning method. The extracted pre-orthodonticfeature vectors Fpre (i=1, 2, . . . , p) and post-orthodontic featurevectors Fpost (i=1, 2, . . . , p) are incorporated, as case data foreach patient, to the database 21. Furthermore, when the pre-orthodonticfeature vectors Fpre(i) and post-orthodontic feature vectors Fpost(i) ofprevious patients are already provided to the database 21, thecalculation processing device 10 may carry out the subsequent processingin view of the vector data of the database 21 without performingprocessing for extracting those feature vectors.

A means for classifying case class 180 is to carry out clusteringprocessing of a group of pre-orthodontic feature vectors Fpre(i) ofprevious patients, calculate plural (for example, number of N) clustercenters Gpre (l=1, 2, . . . , N) for the pre-orthodontic feature vectorsFpre(i), and have classification into each case class CL (l=1, 2, . . ., N) (step S32). As the clustering processing, a general vectorquantization method like Lloyd method and k-means method can be used(see, FIGS. 8 and 9).

According to k-means method, for example, clustering processing offeature vectors Fpre(i) can be carried out as described below. First,number N of the primary clusters is arbitrarily set, and virtualclusters CL*(l=1, 2, . . . , N) are allotted for n-dimensional vectorspace (n is the number of feature variables v). Next, according tocalculation of the average of feature vectors Fpre(i) belonging to eachprimary cluster CL*(l), primary cluster centers G*(l=1, 2, . . . , N)are obtained. Then, the distances D*(l, i)=|G*(l)−Fpre(i)|, which arebetween each of obtained N centers G*(l) and all feature vectorsFpre(i), are obtained. Herein, the “distance” between vectors can be anyof Euclid distance or Manhattan distance.

Next, primary cluster center G*(l) present at the shortest distance whenseen from each feature vector Fpre(i) is identified, and secondaryclusters CL**(l), which have as elements a group of feature vectorsFpre(i) commonly having the shortest distance center G*(l), arere-organized. Then, also for secondary clusters CL**(l), secondarycluster centers G**(l) are obtained, and tertiary cluster centersG***(l) are obtained from a group of feature vectors Fpre(i) at theshortest distance. Feature vectors Fpre(i) of each patient can beclassified for N clusters (case classes) CL (l=1, 2, . . . , N) whichhave been assigned by repeating this cycle of clusters re-organization(see, FIG. 9).

It is possible that optimization processing of clusters (that is, caseclasses) number is subsequently carried out according to the algorithmshown below.

First, number N of clusters to be a candidate is set within a reasonablerange like N=3, 4, . . . , 12, and centers GN (l=1, 2, . . . , N) ofeach cluster classified for each cluster number are obtained. Theminimum distance DN(l)min of a distances DN(l,j) between each centerGN(l), which is obtained from each cluster number N=3, 4, . . . , 12,and feature vectors Fpre(i) belonging to cluster of each center GN(l),is obtained by using the mathematical formula (1).

$\begin{matrix}{\left\lbrack {{Math}.\mspace{14mu} 1} \right\rbrack \mspace{644mu}} & \; \\{{D_{N}(l)}_{\min} = {\min\limits_{j}\left( {D_{N}\left( {l,j} \right)} \right)}} & (1)\end{matrix}$

The cluster-to-cluster distance DN, which is an average of the minimumdistance of each cluster, is obtained by using the mathematical formula(2).

$\begin{matrix}{\left\lbrack {{Math}.\mspace{14mu} 2} \right\rbrack \mspace{644mu}} & \; \\{D_{N} = {\left( {\sum\limits_{l = 1}^{N}{D_{N}(l)}_{\min}} \right)/N}} & (2) \\{{\Delta \; D_{N}} = {D_{N + 1} - D_{N}}} & (3)\end{matrix}$

For cluster candidates, for example, N=3, 4, . . . , 12, each of thecluster-to-cluster distance D3, D4, . . . , D12 are obtained, and N+1that is obtained by adding 1 to N at which the variance ΔDN is at themaximum can be determined as Nopt which is the optimum cluster number.

By carrying out the above clustering processing by a means forclassifying case class 180, plural cluster centers Gpre(l) can becalculated from case data of previous patients who have already receivedan orthodontic dental treatment, and they can be classified into eachcase class CL(l) (step S32).

Furthermore, according to this embodiment, a means for extractingfeature vector 110 may include a means for extracting pre-orthodonticcephalo feature vector 114. In that case, a means for extractingpre-orthodontic cephalo feature vector 114 is to extract, from cephalodata acquired before an orthodontic treatment of a previous patient whohas already received an orthodontic dental treatment, pre-orthodonticcephalo feature vectors Cpre(i) which have multi-dimensional featurevariables as elements (step S33). In addition, regarding the processingof step S32, it is also possible that, by a means for classifying caseclass 180, clustering processing is carried out for a group of extendedfeature vectors V(i)=[Fpre(i), Cpre(i)] in which the pre-orthodonticfeature vectors Fpre(i) and the pre-orthodontic cephalo feature vectorsCpre(i) of a previous patient are composited, and classifying it to eachcase class CL(l).

Furthermore, a means for extracting feature vector 110 may additionallyinclude a means for extracting post-orthodontic cephalo feature vector115. In that case, a means for extracting post-orthodontic cephalofeature vector 115 is to extract, from cephalo data acquired after anorthodontic treatment of a previous patient who has already received anorthodontic dental treatment, post-orthodontic cephalo feature vectorsCpost(i) which have multi-dimensional feature variables as elements(step S34). In addition, regarding the processing of step S32, it isalso possible that, by a means for classifying case class 180,clustering processing is carried out for a group of extended featurevectors V(i) in which the pre-orthodontic feature vectors Fpre(i),pre-orthodontic cephalo feature vectors Cpre(i), and post-orthodonticcephalo feature vectors Cpost(i) of previous patients are composited. Inthat case, the feature vector V(i) includes the differenceCpre(i)−Cpost(i) between pre-orthodontic and post-orthodontic cephalofeature vectors in a vector element. More specifically, it is preferablethat V(i)=[Fpre(i), Cpre(i), Cpre(i)−Cpost(i)].

The cephalo data of previous patients, or pre-orthodontic cephalofeature vectors Cpre(i) and post-orthodontic cephalo feature vectorsCpost(i) that are extracted from them are incorporated to the database21. When the pre-orthodontic cephalo feature vectors Cpre(i) andpost-orthodontic cephalo feature vectors Cpost(i) are already providedas case data to the database 21, the calculation processing device 10may carry out the subsequent processing in view of the data of thedatabase 21 without performing the calculation processing by a means forextracting feature vector 110.

Furthermore, according to the clustering processing of step S32, clustercenters Gpre(l) and the like classified from the case data of previouspatients are incorporated to the database 21. When cluster centersGpre(l) and the like are already provided as case data to the database21, the calculation processing device 10 may carry out only thesubsequent processing according to facial shape prediction of a patientas an evaluation subject without performing the clustering processingdescribed above.

First, a means for extracting subject patient feature vector 113 is toextract an n-dimensional subject patient feature vector Fnew [v1, v2,v3, . . . , vn] which has, as elements, predetermined feature variablesfrom three-dimensional face data (patient shape data) of a patientcontemplating an orthodontic treatment (step S35).

A means for selecting similar case class 190 is to select, from centersGpre(l) of pre-orthodontic feature vectors Fpre(i) belonging to eachcase class which has been classified by clustering processing of casedata of previous patients, a similar case class CL (near) which has avector center having the shortest distance (|Gpre(l)−Fnew|) from thesubject patient feature vector Fnew (step S36; see FIG. 9). In thatcase, the “distance” can be any of Euclid distance or Manhattandistance.

A means for modeling subject patient facial shape 132 is to calculatesubject patient facial shape model (patient shape model) Hnew which isobtained by extracting anatomical feature points from three-dimensionalface data of a patient as an evaluation subject (patient shape data) andnormalizing them with a homologous model, for example (step S37).

Furthermore, a means for modeling case patient facial shape 131 is tocalculate pre-orthodontic facial shape models (pre-treatment similarcase shape models) Hpre (i=i1, i2, . . . , ik) which are obtained bynormalizing the arrangements of the anatomical feature points obtainedfrom pre-orthodontic three-dimensional face data of each case patientbelonging to similar case class CL (near) as selected in step S36 with ahomologous model, for example (step S38). Similarly, a means formodeling case patient facial shape 131 is to calculate post-orthodonticfacial shape models (post-treatment similar case shape models)Hpost(i=i1, i2, . . . , ik) which are obtained by normalizing thearrangements of the anatomical feature points obtained frompost-orthodontic three-dimensional face data of each case patientbelonging to the similar case class CL (near) with a homologous model,for example (step S39).

Herein, as case data, all of the pre-orthodontic facial shape modelsHpre(i) and post-orthodontic facial shape models Hpost(i) calculated forpatients may be incorporated to the database 21. Furthermore, when thepre-orthodontic facial shape models Hpre(i) and post-orthodontic facialshape models Hpost(i) of previous patients are already provided to thedatabase 21, the calculation processing device 10 may carry out thesubsequent processing in view of the model data of the database 21without performing the modeling processing.

Subsequently, a means for typing facial shape 140 is to calculate vectoraverage AVEpre of pre-orthodontic facial shape models Hpre (i=i1, i2, .. . , ik) of each case patient belonging to the selected similar caseclass CL (near) (step S40). Similarly, a means for typing facial shape140 is to calculate vector average AVEpost of post-orthodontic facialshape models Hpost (i=i1, i2, . . . , ik) of each case patient belongingto the selected similar case class CL (near) (step S41).

Subsequently, a means for calculating a predicted facial shape model 150is to calculate facial shape vector average difference AVEpost−AVEpre bysubtracting the vector average AVEpre of pre-orthodontic facial shapemodel from the vector average AVEpost of post-orthodontic facial shapemodel (step S42). In addition, a means for calculating a predictedfacial shape model 150 is to perform calculation for adding facial shapevector average difference AVEpost−AVEpre of each patient of a similarcase class to subject patient facial shape model Hnew of a patient as anevaluation subject, and thus obtaining predicted three-dimensionalpredicted facial shape model Hprd (=Hnew+AVEpost−AVEpre) of a patient asan evaluation subject after an orthodontic treatment (step S43).

A means for reconstructing facial shape 160 is preferably to reconstructa predicted three-dimensional face data by rearranging the anatomicalfeature points of predicted facial shape model Hprd in athree-dimensional face data coordinate system of a patient as anevaluation subject (step S42). Accordingly, it becomes possible todisplay three-dimensionally a lateral view of a face of a patientpredicted after a treatment or the like on the output device 40 likeimage display (step S45), and thus a patient is allowed to fullyunderstand the effect of an orthodontic dental treatment while watchingthe image.

With a means for incorporating case data 170, three-dimensional facedata of a subject patient after the evaluation may be incorporated tothe database 21. Furthermore, with a means for incorporating case data170, pre-orthodontic subject patient feature vector Fnew extracted fromthe three-dimensional face data of a patient as an evaluation subject orsubject patient facial shape model Hnew obtained by modeling theanatomical feature points of three-dimensional face data may beincorporated to the database 21. It is also possible that, with a meansfor incorporating case data 170, the post-orthodontic treatmentthree-dimensional face data of the patient, the post-orthodontic featurevector and/or post-orthodontic facial shape model which is extractedfrom the post-treatment three-dimensional face data are incorporated tothe database 21. By accumulating those data in the database 21,precision of the facial shape prediction after a treatment of a futurepatient can be further enhanced.

According to the method for predicting a facial shape or the system forpredicting a facial shape of the embodiment described above, athree-dimensional facial shape of a patient after an orthodontic dentaltreatment can be predicted conveniently and highly precisely by anarithmetic calculation processing. Furthermore, because thethree-dimensional facial shape of a patient to be a subject can beevaluated quantitatively, they can contribute to the suitabledetermination of a treatment plan for the patient in view of previouscases.

The method for predicting a facial shape and the system for predicting afacial shape according to the present invention can be also used, otherthan the orthodontic dental treatment, for a surgical treatment or anaesthetic improvement for a patient with jaw deformity, and the like.Furthermore, they can be used for prediction when a maxillofacialsurgery (including oral surgery and plastic surgery) operation iscarried out alone, or in combination with an orthodontic dentaltreatment or a jaw prosthetic treatment. They are also expected to beapplied for predicting a change in facial shape associated with ageing.

Third Embodiment

Next, the third embodiment of the present invention is explained.According to the third embodiment, a method for predicting convenientlyand also highly precisely a breast shape by arithmetic calculationprocessing of a three-dimensional breast shape after breastcancer-related mastectomy (operation for removing breast cancer), forexample, and a system used for the method are provided. In FIG. 10, abrief constitution of the system for predicting a breast shape accordingto this embodiment is exemplified.

The calculation processing device 10 is connected with the data storage20 with large capacity, the input device 30, and the output device 40.Stored In the data storage 20 is a group of case data prepared as thedatabase 21 including pre-operative and post-operative three-dimensionalbreast shape data (case shape data) which have been photographed fromseveral previous breast cancer patients, feature vectors (case featurevectors) Fpre(i), Fpost(i) that are extracted from pre-operative andpost-operative three-dimensional breast shape data, andthree-dimensional shape models (case shape model) Hpre(i), Hpost(i)obtained by normalizing pre-operative and post-operativethree-dimensional breast shape data.

The calculation processing device 10 is a computer device which performscalculation processing based on the data that are memorized in the datastorage 20 or the database 21. The input device 30 includes amanipulation input device like keyboard, mouse, touch panel, or thelike, for example. Furthermore, the input device 30 can be a devicewhich has a function of inputting data that have been acquired orprocessed by another system to the calculation processing device 10 viaan information memory medium or a network. The output device 40 includesan image display for three-dimensional visual display of predictedbreast shape data, for example. Furthermore, for providing intermediatedata like feature vector processed by the calculation processing device10 or evaluation data like predicted breast shape model to anothersystem, the output device 40 can be a memory drive for memorizing thosedata in an information memory medium or a communication output devicefor outputting those data to an outside via a network.

Furthermore, the system for predicting a breast shape is constitutedsuch that photography data of a patient or three-dimensional data thatare taken in a hospital examination room or the like are supplied to thecalculation processing device 10, either directly or via the datastorage 20 or the database 21. It is acceptable that those data of apatient are inputted to the input device 30 via an information memorymedium, or inputted to the system via a hospital network, for example.The system for predicting a breast shape may also include, as aconstitutional element, a digital camera 61 for taking a picture of abreast as a disorder area of a patient, a three-dimensional camera 62for acquiring three-dimensional breast shape data of a patient, or ageneral three-dimensional measurement device like three-dimensionalscanner or laser profiler.

The calculation processing device 10 according to the third embodimentis provided with, as a means for calculation processing to be achievedby the calculation processing, a means for extracting feature vector210, a means for selecting similar case patient 220, a means formodeling breast shape 230, a means for typing breast shape 240, a meansfor calculating predicted shape model 250, a means for reconstructing abreast shape 260, a means for incorporating case data 270, a means forclassifying case class 280, and a means for selecting similar case class290.

A means for extracting feature vector 210 includes a means forextracting pre-operative feature vector 211, a means for extractingpost-operative feature vector 212, and a means for extracting subjectpatient feature vector 213.

A means for modeling breast shape 230 includes a means for modeling casepatient breast shape 231 and a means for modeling subject patient breastshape 232.

The method for predicting a breast shape carried out based on theaforementioned system is explained in detail in view of the flowchart ofFIG. 1. First, a means for extracting pre-operative feature vector 211is to extract n-dimensional pre-operative feature vectors (pre-treatmentcase feature vectors) Fpre(1) [v1, v2, v3, . . . , vn], Fpre(2) [v1, v2,v3, . . . , vn], . . . , Fpre(p) [v1, v2, v3, . . . , vn], which have,as elements, plural feature variables that are set in advance, frompre-operative three-dimensional breast shape data (case shape databefore treatment) of p patients who have already received a breastcancer treatment (step S51).

According to the same processing as described above, a means forextracting post-operative feature vector 212 extracts n-dimensionalpost-operative feature vectors (post-treatment case feature vectors)Fpost(1) [v1, v2, v3, . . . , vn], Fpost(2) [v1, v2, v3, . . . , vn], .. . , Fpost(p) [v1, v2, v3, . . . , vn] from post-operativethree-dimensional breast shape data (case shape data after treatment) ofthe same p patients.

It is also possible that pre-operative and post-operative featurevectors Fpre(i) and Fpost(i) are extracted by using a deep learningmethod. The extracted feature vectors Fpre (i=1, 2, . . . , p) and Fpost(i=1, 2, . . . , p) are incorporated, as case data for each patient, tothe database 21.

A means for classifying case class 280 is to carry out clusteringprocessing of a group of pre-operative feature vectors Fpre(i) of thosecase patients. In that case, case feature vectors Fpre(i) obtained afterperforming the same treatment as the subject patient are calculated toplural (for example, number of N) cluster centers Gpre (l=1, 2, . . . ,N), and classified into each case class CL (l=1, 2, . . . , N) (stepS52). As the clustering processing, a general vector quantization methodlike Lloyd method and k-means method can be used as described in theabove (see, FIGS. 8 and 9). Furthermore, for this clustering processing,it is more preferable to supply information regarding breast shape orremoved tissue amount in combination in addition to the describedequation.

According to the clustering processing of step S52, cluster centersGpre(l) and the like that are classified from case data of previouspatients are incorporated to the database 21. Furthermore, when clustercenters Gpre(l) and the like are already provided as case data to thedatabase 21, the calculation processing device 10 may carry out thesubsequent processing for predicting a breast shape of a patient as anevaluation subject without performing the clustering processing of stepS52.

A means for extracting subject patient feature vector 213 extracts an-dimensional subject patient feature vector Fnew [v1, v2, v3, . . . ,vn], which has a predetermined feature variables as elements, fromthree-dimensional breast shape data (patient shape data) of a patient asan evaluation subject who is contemplating a treatment (step S53).

A means for selecting similar case class 290 is to select, from centersGpre(l) of pre-orthodontic feature vectors Fpre(i) belonging to eachcase class which has been classified by clustering processing of casedata of previous patients, a similar case class CL (near) which has avector center having the shortest distance (|Gpre(l)−Fnew|) from thesubject patient feature vector Fnew (step S54). In that case, the“distance” can be any of Euclid distance or Manhattan distance.

A means for modeling subject patient breast shape 232 is to calculatesubject patient breast shape model (patient shape model) Hnew which isobtained by extracting anatomical feature points from three-dimensionalbreast shape data of a patient as an evaluation subject and normalizingthem with a homologous model, for example (step S55).

Furthermore, a means for modeling case patient breast shape 231 is tocalculate pre-operative breast shape models (pre-treatment similar caseshape models) Hpre (i=i1, i2, . . . , ik) which are obtained bynormalizing the arrangements of the anatomical feature points obtainedfrom pre-operative three-dimensional breast shape data of each casepatient belonging to the similar case class CL (near) as selected instep S54 with a homologous model, for example (step S56). Similarly, ameans for modeling case patient breast shape 231 is to calculatepost-operative breast shape models (post-treatment similar case shapemodels) Hpost(i=i1, i2, . . . , ik) which are obtained by normalizingthe arrangements of the anatomical feature points obtained frompost-operative three-dimensional breast shape data of each case patientbelonging to the similar case class CL (near) with a homologous model,for example (step S57).

Herein, as case data, all of the pre-operative breast shape modelsHpre(i) and post-operative breast shape models Hpost(i) calculated forpatients may be incorporated to the database 21. Furthermore, when thepre-operative breast shape models Hpre(i) and post-operative breastshape models Hpost(i) of previous patients are already provided to thedatabase 21, the calculation processing device 10 may carry out thesubsequent processing in view of the model data of the database 21without performing the modeling processing.

Subsequently, a means for typing breast shape 240 is to calculate vectoraverage AVEpre of pre-operative breast shape models Hpre (i=i1, i2, . .. , ik) of each case patient belonging to the selected similar caseclass CL (near) (step S58). Similarly, a means for typing breast shape240 is to calculate vector average AVEpost of post-operative breastshape models Hpost (i=i1, i2, . . . , ik) of each case patient belongingto the selected similar case class CL (near) (step S59).

Subsequently, a means for calculating predicted shape model 250 is tocalculate breast shape vector average difference AVEpost−AVEpre bysubtracting the vector average AVEpre of pre-operative breast shapemodel from the vector average AVEpost of post-operative breast shapemodel (step S60). In addition, a means for calculating predicted shapemodel 250 is to perform calculation for adding breast shape vectoraverage difference AVEpost−AVEpre of each patient of a similar caseclass to the breast shape model Hnew of a patient as an evaluationsubject, and thus obtaining three-dimensional predicted breast shapemodel Hprd (=Hnew+AVEpost−AVEpre) of a patient as an evaluation subjectafter operation (step S61).

A means for reconstructing a breast shape 260 is to reconstruct apredicted three-dimensional breast data by rearranging the anatomicalfeature points of predicted breast shape model Hprd in athree-dimensional breast shape data coordinate system of a patient as anevaluation subject (step S62). Accordingly, it becomes possible todisplay three-dimensionally a breast shape of a patient predicted afteran operation on the output device 40 like image display (step S63).Accordingly, the patient is allowed to fully understand the effect ofthe treatment.

With a means for incorporating case data 270, three-dimensional breastshape data of a subject patient after the evaluation may be incorporatedto the database 21. Furthermore, with a means for incorporating casedata 270, pre-operative subject patient feature vector Fnew extractedfrom the three-dimensional breast shape data of a patient as anevaluation subject or subject patient breast shape model Hnew obtainedby modeling the anatomical feature points of three-dimensional breastshape data may be incorporated to the database 21. It is also possiblethat, with a means for incorporating case data 270, thethree-dimensional breast shape data of the patient after a treatment,the breast shape feature vector and/or breast shape normalized model areincorporated to the database 21. By accumulating those data in thedatabase 21, precision of the breast shape prediction after a treatmentof a future patient can be further enhanced.

According to the method for predicting a breast shape or the system forpredicting a breast shape which has been explained in the above, thethree-dimensional breast shape of a patient after a treatment can beconveniently and highly precisely predicted based on an arithmeticcalculation processing. Furthermore, because the three-dimensionalbreast shape of a patient to be a subject can be quantitativelyevaluated, they can contribute to the suitable determination of atreatment plan for the patient in view of previous cases.

EXPLANATIONS OF LETTERS OR NUMERALS

-   -   10 Calculation processing device    -   21 Database    -   40 Output device    -   62 Three-dimensional camera    -   110 Means for extracting feature vector    -   120 Means for selecting similar case patient    -   130 Means for modeling facial shape    -   140 Means for typing facial shape    -   150 Means for calculating predicted facial shape model    -   160 Means for reconstructing facial shape    -   170 Means for incorporating case data    -   180 Means for classifying case class    -   190 Means for selecting similar case class    -   210 Means for extracting feature vector    -   220 Means for selecting similar case patient    -   230 Means for modeling breast shape    -   240 Means for typing breast shape    -   250 Means for calculating predicted shape model    -   260 Means for reconstructing breast shape    -   270 Means for incorporating case data    -   280 Means for classifying case class    -   290 Means for selecting similar case class    -   Fnew Patient feature vector    -   Fpre, Fpost Case feature vector    -   Hnew Patient shape model    -   Hpre, Hpost Case shape model

1-14. (canceled)
 15. A method for predicting a shape of a human bodyafter a treatment by arithmetic calculation processing comprising: astep of extracting multi-dimensional case feature vectors frompre-treatment three-dimensional case shape data obtained from aplurality of case patients who have received the treatment; a step ofextracting a multi-dimensional patient feature vector from thethree-dimensional patient shape data obtained from a patient as anevaluation subject; a step of calculating a patient shape modelnormalized based on the three-dimensional patient shape data of thepatient as an evaluation subject; a step of selecting similar casefeature vectors of a plurality of case patients that are similar to thepatient feature vector from the case feature vectors; a step ofcalculating a plurality of pre-treatment and post-treatment similar caseshape models that are normalized based on three-dimensional case shapedata corresponding to the selected similar case feature vectors of theplurality of case patients; a step of calculating a vector average ofthe plurality of similar case shape models; a step of calculating avector average difference between the pre-treatment similar case shapemodels and the post-treatment similar case shape models; and a step ofmodifying the patient shape model of the patient as an evaluationsubject with the vector average difference and calculating a shape modelas predicted after the treatment of the patient as an evaluationsubject.
 16. A method for predicting a facial shape after an orthodontictreatment comprising: a step of extracting multi-dimensionalpre-orthodontic feature vectors Fpre(i) which have, as elements, aplurality of feature variables that are set in advance for a humanfacial shape based on pre-orthodontic three-dimensional face data of aplurality of patients who have received an orthodontic treatment; a stepof extracting a multi-dimensional subject patient feature vector Fnewhaving the feature variables as elements based on three-dimensional facedata of a new patient contemplating an orthodontic treatment (patient asan evaluation subject); a step of selecting, in a predetermined casenumber k, case patients having the pre-orthodontic feature vectorsFpre(i) that are related to a plurality of the patients who havereceived the orthodontic treatment in which the selection is made inorder of the shorter distance from the subject patient feature vectorFnew; a step of calculating pre-orthodontic facial shape models Hpre(i=i1, i2, . . . , ik) as facial shape models of the each case patientin which the arrangements of feature points obtained frompre-orthodontic three-dimensional face data of the each selected casepatient are normalized; a step of calculating post-orthodontic facialshape models Hpost (i=i1, i2, . . . , ik) of the each case patient inwhich the arrangements of feature points obtained from post-orthodonticthree-dimensional face data of the each selected case patient arenormalized; a step of calculating a vector average AVEpre ofpre-orthodontic facial shape models Hpre (i=i1, i2, . . . , ik) of theeach selected case patient; a step of calculating a vector averageAVEpost of post-orthodontic facial shape models Hpost (i=i1, i2, . . . ,ik) of the each selected case patient; a step of calculating a facialshape vector average difference AVEpost−AVEpre that is obtained bysubtracting the vector average AVEpre of pre-orthodontic facial shapemodels from the vector average AVEpost of post-orthodontic facial shapemodels; a step of calculating a subject patient facial shape model Hnewas a facial shape model of the patient in which the arrangement offeature points obtained from the three-dimensional face data of thepatient as an evaluation subject is normalized; and a step ofcalculating a three-dimensional predicted facial shape model Hprd aspredicted after an orthodontic treatment of the patient as an evaluationsubject by modifying the facial shape model Hnew of the patient as anevaluation subject with the facial shape vector average differenceAVEpost−AVEpre of the each selected case patient.
 17. A method forpredicting a facial shape after an orthodontic treatment comprising: astep of extracting multi-dimensional pre-orthodontic feature vectorsFpre(i) which have, as elements, a plurality of feature variables thatare selected in advance for a human facial shape based onpre-orthodontic three-dimensional face data of a plurality of patientswho have received an orthodontic treatment; a step of calculating, forclassification of case data of the plurality of patients, case classesclassified into a plurality of classes by carrying out clusteringprocessing of a group of pre-orthodontic feature vectors Fpre(i) of theplurality of patients, and cluster centers Gpre(l) of the each caseclass; a step of extracting a multi-dimensional subject patient featurevector Fnew having the feature variables as elements based onthree-dimensional face data of a new patient contemplating anorthodontic treatment (patient as an evaluation subject); a step ofselecting, from a plurality of the cluster centers Gpre(l) after theclustering, a similar case class having a cluster center that has theshortest distance from the subject patient feature vector Fnew; a stepof calculating pre-orthodontic facial shape models Hpre (i=i1, i2, . . ., ik) as facial shape models of the each case patient in which thearrangements of feature points obtained from pre-orthodonticthree-dimensional face data of the each selected case patient belongingto the similar case class are normalized; a step of calculatingpost-orthodontic facial shape models Hpost (i=i1, i2, . . . , ik) of theeach case patient in which the arrangements of feature points obtainedfrom post-orthodontic three-dimensional face data of the each selectedcase patient belonging to the similar case class are normalized; a stepof calculating a vector average AVEpre of pre-orthodontic facial shapemodels Hpre (i=i1, i2, . . . , ik) of each selected case patientbelonging to the similar case class; a step of calculating a vectoraverage AVEpost of post-orthodontic facial shape models Hpost (i=i1, i2,. . . , ik) of each selected case patient belonging to the similar caseclass; a step of calculating a facial shape vector average differenceAVEpost−AVEpre that is obtained by subtracting the vector average AVEpreof pre-orthodontic facial shape models from the vector average AVEpostof post-orthodontic facial shape models; a step of calculating a subjectpatient facial shape model Hnew as a facial shape model of the patientin which the arrangement of feature points obtained from thethree-dimensional face data of the patient as an evaluation subject isnormalized; and a step of calculating a three-dimensional predictedfacial shape model Hprd as predicted after an orthodontic treatment ofthe patient as an evaluation subject by modifying the facial shape modelHnew of the patient as an evaluation subject with the facial shapevector average difference AVEpost−AVEpre of the each selected casepatient belonging to the similar case class.
 18. The method forpredicting a facial shape according to claim 17, further comprising: astep of extracting multi-dimensional pre-orthodontic cephalo featurevectors Cpre(i) which have, as elements, a plurality of featurevariables that are set in advance for a human bone shape based onpre-orthodontic head part X ray images of the plurality of patients whohave received an orthodontic treatment, wherein, in the step forclassifying case data of the plurality of patients, the clusteringprocessing is carried out for a group of extended feature vectors V(i)in which the pre-orthodontic feature vectors Fpre(i) and thepre-orthodontic cephalo feature vectors Cpre(i) of the plurality ofpatients are composited.
 19. The method for predicting a facial shapeaccording to claim 18, further comprising: a step of extractingmulti-dimensional pre-orthodontic cephalo feature vectors Cpre(i) whichhave, as elements, a plurality of feature variables that are set inadvance for a human bone shape based on pre-orthodontic head part X rayimages of the plurality of patients who have received an orthodontictreatment; and a step of extracting multi-dimensional post-orthodonticcephalo feature vectors Cpost(i) based on post-orthodontic head part Xray images of the plurality of patients, wherein, in the step forclassifying case data of the plurality of patients, the clusteringprocessing is carried out for a group of extended feature vectors V(i)in which the pre-orthodontic feature vectors Fpre(i), thepre-orthodontic cephalo feature vectors Cpre(i), and thepost-orthodontic cephalo feature vectors Cpost(i) of the plurality ofpatients are composited.
 20. The method for predicting a facial shapeaccording to claim 19, wherein the extended feature vector V(i) isV(i)=[Fpre(i), Cpre(i), Cpre(i)−Cpost(i)] including, as an vectorelement, the cephalo feature vector difference Cpre(i)−Cpost(i) betweenthe pre-orthodontic treatment and post-orthodontic treatment.
 21. Themethod for predicting a facial shape according to claim 16, furthercomprising a step of incorporating feature vectors and/or facial shapemodels obtained from the three-dimensional face data of the patient asan evaluation subject to a database.
 22. A system for predicting afacial shape after an orthodontic treatment provided with a database anda calculation processing device for carrying out calculation processingbased on data memorized in the database, wherein the database memorizesin advance at least: multi-dimensional pre-orthodontic feature vectorsFpre(i) which have been extracted having, as elements, a plurality offeature variables that are set in advance for a human facial shape basedon pre-orthodontic three-dimensional face data of a plurality ofpatients who have received an orthodontic treatment; pre-orthodonticfacial shape models Hpre(i) in which the arrangements of feature pointsobtained from pre-orthodontic three-dimensional face data of theplurality of patients are normalized; and post-orthodontic facial shapemodels Hpost(i) in which the arrangements of feature points obtainedfrom post-orthodontic three-dimensional face data of the plurality ofpatients are normalized, and the calculation processing device isprovided with: a means for extracting feature vector to extract amulti-dimensional subject patient feature vector Fnew having the featurevariables as elements based on three-dimensional face data of a newpatient contemplating an orthodontic treatment (patient as an evaluationsubject); a means for selecting similar case patient to select, in apredetermined case number k, case patients having the pre-orthodonticfeature vectors Fpre(i) in order of the shorter distance from thesubject patient feature vector Fnew; a means for typing facial shape tocalculate a vector average AVEpre of pre-orthodontic facial shape modelsHpre (i=i1, i2, . . . , ik) of the each selected case patient andcalculating a vector average AVEpost of post-orthodontic facial shapemodels Hpost (i=i1, i2, . . . , ik) of the each selected case patient; ameans for modeling facial shape to calculate a subject patient facialshape model Hnew as a facial shape model of the patient in which thearrangement of feature points obtained from the three-dimensional facedata of the patient as an evaluation subject is normalized; and a meansfor calculating a predicted facial shape model to calculate athree-dimensional predicted facial shape model Hprd as predicted afteran orthodontic treatment of the patient as an evaluation subject bycalculating a facial shape vector average difference AVEpost−AVEpre thatis obtained by subtracting the vector average AVEpre of pre-orthodonticfacial shape models from the vector average AVEpost of post-orthodonticfacial shape models, and modifying the facial shape model Hnew of thepatient as an evaluation subject with the facial shape vector averagedifference AVEpost−AVEpre of the each selected case patient.
 23. Asystem for predicting a facial shape after an orthodontic treatmentprovided with a database and a calculation processing device forcarrying out calculation processing based on data memorized in thedatabase wherein the database memorizes in advance at least:multi-dimensional pre-orthodontic feature vectors Fpre(i) which havebeen extracted having, as elements, a plurality of feature variablesthat are set in advance for a human facial shape based onpre-orthodontic three-dimensional face data of a plurality of patientswho have received an orthodontic treatment; pre-orthodontic facial shapemodels Hpre(i) in which the arrangements of feature points obtained frompre-orthodontic three-dimensional face data of the plurality of patientsare normalized; and post-orthodontic facial shape models Hpost(i) inwhich the arrangements of feature points obtained from post-orthodonticthree-dimensional face data of the plurality of patients are normalized,and the calculation processing device is provided with: a means forclassifying case class to calculate a case class classified into aplurality of classes by carrying out clustering processing of a group ofpre-orthodontic feature vectors Fpre(i) of the plurality of patients,and cluster centers Gpre(l) of the each case class; a means forextracting feature vector to extract a multi-dimensional subject patientfeature vector Fnew having the feature variables as elements based onthree-dimensional face data of a new patient contemplating anorthodontic treatment (patient as an evaluation subject); a means forselecting similar case class to select, from a plurality of the clustercenters Gpre(l) after the clustering, a similar case class having acluster center that has the shortest distance from the subject patientfeature vector Fnew; a means for typing facial shape to calculate avector average AVEpre of pre-orthodontic facial shape models Hpre (i=i1,i2, . . . , ik) of the each selected case patient belonging to thesimilar case class and to calculate a vector average AVEpost ofpost-orthodontic facial shape models Hpost (i=i1, i2, . . . , ik) of theeach selected case patient belonging to the similar case class; a meansfor modeling facial shape to calculate a subject patient facial shapemodel Hnew as a facial shape model of the patient in which thearrangement of feature points obtained from the three-dimensional facedata of the patient as an evaluation subject is normalized; and a meansfor calculating a predicted facial shape model to calculate athree-dimensional predicted facial shape model Hprd as predicted afteran orthodontic treatment of the patient as an evaluation subject bycalculating a facial shape vector average difference AVEpost−AVEpre thatis obtained by subtracting the vector average AVEpre of pre-orthodonticfacial shape models from the vector average AVEpost of post-orthodonticfacial shape models, and modifying the facial shape model Hnew of thepatient as an evaluation subject with the facial shape vector averagedifference AVEpost−AVEpre of the each selected case patient belonging tothe similar case.
 24. The system for predicting a facial shape accordingto claim 23, wherein the database further memorizes in advancemulti-dimensional pre-orthodontic cephalo feature vectors Cpre(i) whichhave been extracted having, as elements, a plurality of featurevariables that are set in advance for a human bone shape based onpre-orthodontic head part X ray images of the plurality of patients whohave received an orthodontic treatment, and the means for classifyingcase class is to perform clustering processing for a group of extendedfeature vectors V(i) in which the pre-orthodontic feature vectorsFpre(i) and the pre-orthodontic cephalo feature vectors Cpre(i) of theplurality of patients are composited.
 25. The system for predicting afacial shape according to claim 24, wherein the database furthermemorizes in advance multi-dimensional post-orthodontic cephalo featurevectors Cpost(i) which have been extracted based on post-orthodontichead part X ray images of the plurality of patients who have received anorthodontic treatment, and the means for classifying case class is toperform clustering processing for a group of extended feature vectorsV(i) in which the pre-orthodontic feature vectors Fpre(i), thepre-orthodontic cephalo feature vectors Cpre(i), and thepost-orthodontic cephalo feature vectors Cpost(i) of the plurality ofpatients are composited.
 26. The system for predicting a facial shapeaccording to claim 25, wherein the extended feature vector V(i) isV(i)=[Fpre(i), Cpre(i), Cpre(i)−Cpost(i)] including, as an vectorelement, the cephalo feature vector difference Cpre(i)−Cpost(i) betweenthe pre-orthodontic treatment and post-orthodontic treatment.
 27. Thesystem for predicting a facial shape according to claim 22, furthercomprising a means for incorporating case data to incorporate a featurevector and/or a facial shape model obtained from a three-dimensionalface data of the patient as an evaluation subject to a database.
 28. Amethod for predicting a breast shape after a treatment comprising: astep of extracting multi-dimensional pre-operative feature vectorsFpre(i) which have, as elements, a plurality of feature variables thatare selected in advance based on pre-operative three-dimensional breastshape data of a plurality of patients who have received an operationaltreatment; a step of calculating case classes classified into aplurality of classes by carrying out clustering processing of a group ofpre-operative feature vectors Fpre(i) of the plurality of patients, andcluster centers Gpre(l) of the each case class; a step of extracting amulti-dimensional subject patient feature vector Fnew having the featurevariables as elements based on three-dimensional breast shape data of apatient contemplating the treatment; a step of selecting, from aplurality of the cluster centers Gpre(l) after the clustering, a similarcase class having a cluster center that has the shortest distance fromthe subject patient feature vector Fnew; a step of calculatingpre-operative breast shape models Hpre (i=i1, i2, . . . , ik) as abreast shape model of the each case patient in which the arrangements offeature points obtained from pre-operative three-dimensional breastshape data of the each selected case patient belonging to the similarcase class are normalized; a step of calculating post-operative breastshape models Hpost (i=i1, i2, . . . , ik) of the each case patient inwhich the arrangements of feature points obtained from post-operativethree-dimensional breast shape data of the each selected case patientbelonging to the similar case class are normalized; a step ofcalculating a vector average AVEpre of pre-operative breast shape modelsHpre (i=i1, i2, . . . , ik) of each selected case patient belonging tothe similar case class; a step of calculating a vector average AVEpostof post-operative breast shape models Hpost (i=i1, i2, . . . , ik) ofeach selected case patient belonging to the similar case class; a stepof calculating a breast shape vector average difference AVEpost−AVEprethat is obtained by subtracting the vector average AVEpre ofpre-operative breast shape models from the vector average AVEpost ofpost-operative breast shape models; a step of calculating a subjectpatient breast shape model Hnew as a breast shape model of the patientin which the arrangement of feature points obtained from thethree-dimensional breast shape data of the patient as an evaluationsubject is normalized; and a step of calculating a three-dimensionalpredicted breast shape model Hprd as predicted after operation bymodifying the breast shape model Hnew of the patient as an evaluationsubject with the breast shape vector average difference AVEpost−AVEpreof the each selected case patient belonging to the similar case class.29. The method for predicting a facial shape according to claim 17,further comprising a step of incorporating feature vectors and/or facialshape models obtained from the three-dimensional face data of thepatient as an evaluation subject to a database.
 30. The method forpredicting a facial shape according to claim 18, further comprising astep of incorporating feature vectors and/or facial shape modelsobtained from the three-dimensional face data of the patient as anevaluation subject to a database.
 31. The method for predicting a facialshape according to claim 19, further comprising a step of incorporatingfeature vectors and/or facial shape models obtained from thethree-dimensional face data of the patient as an evaluation subject to adatabase.
 32. The method for predicting a facial shape according toclaim 20, further comprising a step of incorporating feature vectorsand/or facial shape models obtained from the three-dimensional face dataof the patient as an evaluation subject to a database.
 33. The systemfor predicting a facial shape according to claim 23, further comprisinga means for incorporating case data to incorporate a feature vectorand/or a facial shape model obtained from a three-dimensional face dataof the patient as an evaluation subject to a database.
 34. The systemfor predicting a facial shape according to claim 24, further comprisinga means for incorporating case data to incorporate a feature vectorand/or a facial shape model obtained from a three-dimensional face dataof the patient as an evaluation subject to a database.